Papers with theoretical grounding
Causal Activation Steering via Sparse Mediation (2026.findings-eacl)
Copied to clipboard
| Challenge: | a sparse mediation steering approach to control language-model behavior is feasible, says a new study . existing methods that learn dense steering vectors modify thousands of activation dimensions simultaneously . |
| Approach: | They propose a sparse mediation steering approach that learns targeted behavioral interventions via regularized training. |
| Outcome: | The proposed method achieves 97-100% of dense baseline effectiveness across four tasks while using only 10-30% of activation dimensions. |
Can Public Large Language Models Help Private Cross-device Federated Learning? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that public data can be used to improve privacy-utility trade-offs for large and small language models. |
| Approach: | They propose to use large-scale public data to help differentially private FL training . they propose a distribution matching algorithm with theoretical grounding to sample public data close to private data distribution . |
| Outcome: | The proposed method is efficient and effective for training private models by taking advantage of public data. |
AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training (2025.emnlp-main)
Copied to clipboard
| Challenge: | Empirically, AdamS demonstrates strong performance in various tasks . et al., 2023b): AdamS is efficient, efficient, and model-agnostic. |
| Approach: | They propose a model-agnostic alternative to Adam for large language model pretraining and post-training. |
| Outcome: | The proposed method matches memory footprint of SGD with momentum while delivering superior performance. |
BloomVQA: Assessing Hierarchical Multi-modal Comprehension (2024.findings-acl)
Copied to clipboard
Yunye Gong, Robik Shrestha, Jared Claypoole, Michael Cogswell, Arijit Ray, Christopher Kanan, Ajay Divakaran
| Challenge: | Recent advances of machine intelligence solutions have demonstrated tremendous success in a wide range of language and multi-modal tasks over diverse domains. |
| Approach: | They propose a VQA dataset to facilitate comprehensive evaluation of large vision-language models on comprehension tasks. |
| Outcome: | The proposed dataset shows improved accuracy over all comprehension levels and a tendency to bypass visual inputs especially for higher-level tasks. |